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LONG-TERM  GOALS 


For  effective  long-term  passive  acoustic  monitoring  of  today’s  large  data  sets,  automated  algorithms 
must  provide  the  ability  to  detect  and  classify  marine  mammal  vocalizations  and  ultimately,  in  some 
cases,  provide  data  for  estimating  the  population  density  of  the  species  present.  In  recent  years, 
researchers  have  developed  a  number  of  algorithms  for  detecting  calls  and  classifying  them  to  species 
or  species  group  (such  as  beaked  whales).  Algorithms  must  be  robust  in  real  ocean  environments 
where  non-Gaussian  and  non-stationary  noise  sources,  especially  vocalizations  from  similar  species, 
pose  significant  challenges.  In  this  project,  we  are  developing  improved  methods  for  detection, 
classification,  and  localization  of  many  types  of  marine  mammal  sounds. 

OBJECTIVES 

We  are  developing  advanced  real-time  passive  acoustic  marine  mammal  detection,  classification,  and 
localization  methods  using  a  two-pronged  approach:  developing  improved  DCL  algorithms,  and 
developing  standardized  interfaces  and  software. 

First,  we  are  developing,  testing,  and  characterizing  advanced  DCL  algorithms: 

1 .  Echolocation  click  classification.  Algorithms  are  being  developed  and  tested  for  several  species 
of  beaked  whales  and  small  odontocetes. 

2.  Tonal  signal  detection  and  classification.  Algorithms  are  being  tested  for  several  species  of 
mysticetes  and  for  small  odontocetes. 

3.  Multi-sensor  localization.  Algorithms  will  be  developed  and  tested  on  datasets  containing 
sounds  of  both  odontocetes  and  mysticetes. 


Second,  improved  DCL  software  will  be  developed  and  both  existing  and  new  methods  will  be  made 
available  to  users.  The  key  contribution  will  be  the  development  of  four  well-specified  interfaces,  for 
detection,  feature  extraction,  classification,  and  localization. 

APPROACH 

Odontocete  click  detection  and  classification.  A  multi-class  support  vector  machine  (SVM)  classifier 
was  previously  developed  (Jarvis  et  al.  2008).  This  classifier  both  detects  and  classifies  echolocation 
clicks  from  five  species  of  odontocetes,  including  Blainville’s  and  Cuvier’s  beaked  whales,  Risso’s 
dolphins,  short-finned  pilot  whales,  and  spenn  whales.  Here  Moretti’s  group,  particularly  S.  Jarvis,  is 
improving  the  SVM  classifier  by  resolving  confusion  between  species  whose  clicks  overlap  in 
frequency. 

The  current  real-time  system  of  Roch  et  al.  for  odontocete  click  classification  is  based  on  Gaussian 
mixture  models  (GMMs)  using  cepstral  feature  vectors.  Cepstral  feature  vectors  provide  a  compact 
representation  of  the  spectrum  (Rabiner  and  Juang  1993)  that  let  the  system  represent  echolocation 
spectra  using  a  reduced  number  of  coefficients,  providing  a  lower-dimensional  feature  space  than  using 
a  standard  representation  of  the  spectrum.  Issues  were  found  in  odontocete  species  identification  with 
equipment  and  site  differences  through  features  derived  from  echolocation  clicks.  Methods  were 
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developed  to  mitigate  these  differences  and  applied  to  the  2015  Detection,  Classification,  Localization, 
and  Density  Estimation  workshop  (DCLDE)  high-frequency  development  and  evaluation  data. 

Tonal  signal  detection  and  classification.  “Tonal  signal”  is  a  generic  term  for  frequency-modulated 
calls  such  as  mysticete  moans  or  odontocete  whistles.  Methods  for  detecting  and  classifying  such 
sounds  are  being  developed  and  applied  to  both  odontocete  whistles  and  baleen  whale  vocalizations, 
including  minke  ( Balaenoptera  acutorostrata),  blue  (B.  musculus),  and  humpback  ( Megaptera 
novaeangliae)  whales. 

Baleen  whale  vocalizations.  Methods  developed  here  for  baleen  whale  detection  and  classification  are 
based  on  automated  detection  and  classification  of  minke  whale  ‘boing’  vocalizations  using  tonal 
signal  methods  which  have  been  previously  applied  to  US  Navy  hydrophone  data  at  PMRF  (Mellinger 
et  al.  2011;  Martin  et  al.  2013).  The  dominant  spectral  component  (DSC,  described  in  Martin  et  al. 
2013)  is  used  for  detection  of  the  call,  as  this  component  is  typically  the  last  component  detected  at 
long  ranges  (>  30  km).  Minke  boing  call  detection  is  used  here  has  a  first-stage  detection  step  similar 
to  the  tonal  detection  processing  described  by  Mellinger  et  al.  (2011).  A  second  stage  is  also  used 
which  processes  a  frequency  band  from  1320  to  1450  Hz  to  detect  the  onset  frequency-modulation 
(FM)  upsweep  component  of  the  call  to  obtain  a  more  accurate  estimate  of  the  start  time  of  the  call  for 
later  localization  processing.  Then  a  third  stage  calculates  the  frequency  with  high  spectral  resolution 
(0.72  Hz  per  bin)  of  the  DSC  for  each  detected  boing  to  help  associate  calls  from  individuals  and  in 
some  cases  to  help  track  individuals  over  multiple  hours. 

Advanced  localization  algorithms.  The  first  requirement  for  passive  acoustic  localization  of  marine 
mammals  is  the  need  to  associate  the  detection  of  an  individual  signal  as  it  is  received  across  the  array 
of  widely  spaced  hydrophones.  Moretti  is  leading  the  effort  to  develop  a  nearest-neighbor  approach  to 
detection  association.  This  approach  still  uses  TDOA/hyperbolic  methods,  but  will  not  discard  TDOA 
from  pairs  of  detections  when  the  normally  requisite  3  detections  are  not  achieved.  Rather,  detections 
from  a  given  hydrophone  will  be  associated  with  detections  from  all  of  its  nearest  neighbors  and  pair¬ 
wise  TDOAs  will  be  calculated. 

Software  and  interfaces.  An  Application  Programming  Interface  (API)  is  a  specification  of  a  set  of 
procedure  calls  (for  objects,  methods),  data  types  (scalars,  structures,  classes,  etc.),  and  protocols  for 
use  of  the  procedures  and  data  types,  making  it  relatively  simple  for  a  developer  to  add  new  algorithms 
to  an  existing  system.  IshmaeTs  (Mellinger  2001)  interfaces  for  detection  and  localization  comprise  a 
relatively  complex  set  of  object  class  methods  (procedure  calls)  and  data  types;  although  it  is 
standardized,  it  is  hardly  straightforward  or  well-documented.  The  M3R  system  (Morrissey  et  al.  2006) 
has  a  fonnat  for  standardized  data  serving  and  detection  message  passing.  We  are  developing  and 
testing  a  plug-in  architecture,  with  corresponding  APIs,  for  these  systems. 

WORK  COMPLETED 

Meetings,  data  sharing  site,  and  funding: 

(1)  We  have  had  telephone  meetings  approximately  monthly  to  discuss  both  technical  details  and 
project  logistics,  with  a  face-to-face  meeting  on  14-Jul-2015  at  the  DCLDE  workshop. 

(2)  We  established  a  private  Internet-accessible  site  for  sharing  data  sets,  meeting  minutes,  and  code. 
The  site  is  private  since  some  of  the  data,  while  not  classified,  is  considered  sensitive. 
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(3)  We  established  a  SourceForge  site  for  sharing  code  and  data  from  the  project,  with  some  code 
uploaded  now  (http://sourceforge.net/proj ects/marinemammalacousticdcl/). 

(4)  Funding  for  this  year  of  the  project  reached  Navy  and  OSU,  though  the  FY15  funding  only  just 
reached  OSU  (on  9/29/15)  and  subcontract  funds  have  not  been  transferred  to  SDSU  yet. 

RESULTS 

Detection/classification  algorithms:  tonal  sounds.  A  whistle  clustering  manuscript  was  revised  with  a 
new  classification  section  showing  that  shape-related  whistle  features  carry  species-specific 
information  useful  for  species  identification.  The  revised  paper  is  currently  in  review. 

Improvements  were  made  to  the  Silbido  whistle  detector  that  improved  the  precision  rate  from  76.0% 
to  89.7%  with  a  corresponding  penalty  in  recall  of  only  0.1%,  representing  a  57%  decrease  in  false 
positives  generated  by  the  detector  with  trivial  impact  on  correct  detections.  A  manuscript  detailing 
these  results  is  underway.  Also,  as  part  of  the  Silbido  effort  to  reduce  false  positives,  we  constructed  a 
visualization  and  debugging  utility  that  lets  one  examine  how  the  algorithm  is  working  and  provided  a 
level  of  abstraction  to  better  understand  where  the  algorithm  was  making  mistakes  and  let  us  examine 
how  proposed  changes  would  affect  performance.  Having  this  type  of  abstraction  was  a  critical 
component  to  recognizing  that  there  were  problems  due  to  noise  regime  changes  and  that  the  whistle 
graphs  associated  with  false  positive  detections  had  certain  characteristics  that  we  could  exploit. 

Detection/classification  algorithms:  odontocete  clicks.  An  article  (Roch  et  ah,  2015)  was  published 
describing  our  work  on  the  impact  of  equipment  and  site  differences  on  odontocete  species 
identification  through  features  derived  from  echolocation  clicks.  The  article  demonstrates  that  there  are 
significant  performance  impacts  on  species  classification  when  training  data  are  recorded  with 
instruments  of  a  specific  type  or  at  a  specific  location  and  tested  under  non-matching  conditions.  This 
has  significant  impact  on  any  Naval  monitoring  scheme  that  uses  features  from  broadband 
echolocation  clicks.  We  were  also  able  to  demonstrate  that  simple  noise  estimates  can  be  used  to 
reduce  this  penalty  significantly.  Further  research  in  this  area  has  the  potential  to  further  mitigate  for 
train/test  mismatch. 

One  of  the  most  important  things  learned  over  the  last  year  is  that  when  one  moves  to  working  with 
very  large  datasets  that  have  not  been  hand-curated  to  extract  sections  with  data  from  relevant  target 
species,  performance  degrades  quickly.  Our  analysis  of  the  2015  DCLDE  development  data  led  us  to 
recognize  that  sporadic  false  positives  from  echolocation  click  detectors  can  wreak  havoc  on 
performance  and  we  subsequently  developed  filtering  rules  that  remove  low  frequency  sporadic  clicks 
as  well  as  methods  of  identifying  and  removing  instrument  self-noise.  Our  work  with  Kullback-Leibler 
distances  between  encounters  (new  start  N000 14- 15- 1-2299)  enabled  us  to  inspect  characteristics  of 
specific  encounters  to  better  detennine  what  problems  exist,  and  we  are  continuing  work  on  this 
productive  path. 

A  class-specific  support  vector  machine  (CS-SVM)  classifier  (Jarvis  2012)  was  developed  to  classify 
the  foraging  clicks  produced  by  several  species  of  odontocete.  The  current  CS-SVM  classifier 
includes  classes  for  the  foraging  clicks  produced  by  Blainville's  beaked  whale,  Mesoplodon 
densirostris  (Md),  Cuvier's  beaked  whale,  Ziphius  cavirostris  (Zc),  and  spenn  whale,  Physeter 
macrocephalus  (Pm),  as  well  as  a  generic  dolphin  class.  This  classifier  has  been  incorporated  into  the 
Marine  Mammal  Monitoring  on  Navy  Ranges  (M3R)  system  which  is  present  at  all  three  on  the  Navy's 
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major  undersea  range  facilities:  the  Atlantic  Undersea  Test  and  Evaluation  Center  (AUTEC),  the 
Southern  California  Off-shore  Range  (SCORE),  and  the  Pacific  Missile  Range  Facility  (PMRF)  (Jarvis 
etal.  2014). 

Beaked  whales  are  also  known  to  produce  homing  pulses,  termed  buzzes  (Johnson  et  al.  2004),  just 
prior  to  a  prey  capture  attempt.  Therefore  buzz  production  can  be  used  as  a  proxy  for  feeding  behavior. 
However,  beaked  whale  buzzes  are  much  more  difficult  to  detect  than  foraging  clicks.  Buzz  clicks  are 
produced  at  a  much  faster  inter-click  interval,  have  a  different  click  structure  and  a  significantly  lower 
source  level  than  foraging  clicks  (Jarvis  et  al.  2008).  A  CS-SVM  classifier  was  developed  specifically 
for  Md  buzzes.  Taking  into  account  the  much  lower  source  level  than  foraging  clicks,  the  detection 
threshold  of  the  buzz  classifier  was  set  correspondingly  lower.  To  avoid  possible  generation  of  an 
unacceptable  number  of  false  alarms  by  the  lower  threshold,  the  buzz  classifier  is  launched  only  after  a 
Md  foraging  click-train  has  been  detected.  Then  the  buzz  classifier  runs  for  only  30  minutes,  the 
average  vocal  period  of  an  Md  dive. 

Localization  and  tracking.  Two  model-based  localization  methods  were  been  developed  for  baleen 
whale  (minke,  fin,  sei,  Bryde’s,  and  humpback  whales)  localization.  One  uses  detection  start  times  to 
detennine  time  difference  of  arrivals  (Martin  et  al.  2015)  fully  developed  on  this  project,  while  the 
second  method  (Helble  et  al.  2015)  was  developed  specifically  for  sequences  of  humpback  song  units 
detected  via  Generalized  Power  Law  processing  cross-correlated  between  hydrophone  pairs.  Both 
model-based  baleen  whale  localization  algorithms  have  been  transitioned  to  an  Operations  and 
Maintenance  Navy  (OMN)  funded  effort  monitoring  for  marine  mammals  in  the  Hawaii  Range 
Complex  (see  associated  projects). 

The  model-based  localization  method  (using  auto-detector  start  times  rather  than  cross  correlations) 
used  for  minke  whale  boing  calls  has  being  used  to  estimate  the  density  of  calling  minke  whales  in  a 
before-during-after  paradigm,  with  results  showing  highly  statistically  significantly  lower  densities  of 
calling  animals  during  MFAS  training  periods  (Martin  et  al.  2015).  The  density  of  calling  minke 
whales  metric  utilized  was  a  minimum  minke  whale  density  as  corrections  for  the  proportion  of  time 
calling  whales  call  and  the  ratio  of  calling  whales  to  the  rest  of  the  population  (both  currently  unknown 
parameters)  would  increase  the  density  estimate.  Figure  4  (from  Martin  et  al.  2015)  shows  the  number 
of  minke  whales  acoustically  localized  in  the  3,780  km"  study  area  in  1  h  periods  for  available 
recorded  data  for  Feb.  201 1,  2012  and  2013. 

Low-frequency  (<100  Hz)  baleen  whale  localization  has  also  shown  sounds  attributed  to  Bryde’s 
whales  for  several  data  sets  (Aug  2013,  Aug  2014,  Sept  2014,  Oct  2014).  Data  is  being  processed  to 
establish  a  baseline  for  Bryde’s  whale  occurrence  at  PMRF.  Humpback  whale  localizations  are  being 
processed  in  the  fleet  monitoring  effort  for  available  data  sets  using  the  Helble  et  al.  2015  algorithm. 
Efforts  at  processing  recorded  data  between  2011  and  2014  indicate  a  total  of  8 1  humpback  whales 
localized  on  the  offshore  range  hydrophones  (Henderson  et  al.  in  review). 

The  Matlab  kinematic  tracking  code  developed  on  the  ADCL  effort  was  uploaded  to  the  SourceForge 
site  with  a  sample  data  set  of  minke  whale  localizations  from  PMRF.  By  automating  the  process  of 
assigning  calls  to  individuals  in  this  manner,  studies  such  as  Martin  et  al.  2015  can  be  done  with  much 
less  manual  effort.  In  addition,  tracking  is  revealing  interesting  behaviors  such  as  spatio-temporally 
clustered  calls  and  potential  interactions  between  species. 
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Software:  The  architecture  for  writing  detection,  classification,  and  localization  modules  has  been 
completed  and  communication  between  Ishmael  and  PAMGUARD  and  a  test  module  has  been 
established.  The  architecture  provides  a  translation  library  for  each  DCL  platform  supported  that 
marshals  data  into  a  fonnat  that  can  be  shared  with  other  processes.  Modules  run  as  separate  programs 
that  share  a  limited  region  of  memory  with  the  DCL  platform.  This  allows  modules  written  on 
platforms  that  require  separate  processes  (e.g.  Matlab,  R)  to  be  gracefully  handled.  Users  designing 
classification  modules  will  configure  the  DCL  platform  to  send  data  to  their  module  and  make  calls  to 
a  standard  interface  library.  Results  are  sent  back  to  the  DCL  platform  in  a  similar  manner.  The  plug-in 
architecture  has  been  successfully  demonstrated  in  Ishmael  with  a  detector  in  MATLAB.  The 
architecture  now  supports  multiple  languages,  and  we  are  continuing  work  to  refine  the  interface  to 
make  it  as  easy  to  use  as  possible. 

IMPACT/APPLICATIONS 

For  the  Navy,  passive  acoustic  monitoring  (PAM)  provides  a  means  of  long-term  monitoring  of  many 
cetacean  populations,  especially  over  areas  of  high  interest.  Such  areas  are  repeatedly  subjected  to 
Navy  exercises  involving  intense  sounds,  especially  multi-ship  mid-frequency  active  (MFA)  sonar. 
Currently,  required  environmental  monitoring  is  dependent  primarily  on  visual  line  transect  surveys 
that  are  costly  and,  in  the  case  of  aerial  surveys,  significantly  dangerous.  In  both  the  areas  critical  to 
the  Navy  and  in  other  areas  critical  to  marine  mammals,  PAM  is  dependent  on  automated  DCL 
methods.  The  advanced  DCL  algorithms  being  developed  here  will  make  PAM  more  effective  and 
efficient;  the  algorithm  implementations  across  standardized  interfaces  that  handle  both  real-time  and 
pre-recorded  data  streams  from  diverse  platfonns  will  make  them  available  to  Navy  fleet  operators  as 
well  as  the  wider  marine  mammal  research  community. 

TRANSITIONS 

Both  model-based  localization  algorithms  for  baleen  whales  have  been  transitioned  to  COMPACFLT 
Operations  and  Maintenance  Navy  (OMN)-fimded  effort  monitoring  for  marine  mammals  in  the 
Hawaii  Range  Complex  (see  related  projects).  This  has  enabled  processing  thousands  of  hours  of 
multiple  hydrophone  data  sets  to  localize  baleen  species  and  documenting  a  behavioral  response,  in 
terms  of  reductions  in  the  densities  of  boing  calling  minke  whales  in  a  study  area  3,780  km2,  to  the  US 
Navy  mid-frequency  active  sonar  training  activities  at  PMRF  (Martin  et  al  2015). 

The  Ishmael  software  package  is  being  improved  as  part  of  an  effort  funded  by  the  Navy’s  Living 
Marine  Resources  (LMR)  program  to  make  a  large  suite  of  detectors  and  classifiers  written  in 
MATLAB,  many  developed  under  this  grant,  accessible  and  easy  to  load  over  the  Internet.  Ishmael  is 
also  being  connected  with  the  M3R  system  for  use  at  Navy  ranges. 

RELATED  PROJECTS 

“Acoustically-equipped  Ocean  Gliders  for  Environmental  and  Oceanographic  Research”  (N00014-13- 
1-0682)  award  to  D.  Mellinger  and  H.  Klinck.  This  is  a  DURIP  grant  to  purchase  acoustically- 
equipped  Seagliders  that  will  use  automated  detection/classification  methods  developed  here. 

“Autonomous  Passive  Acoustic  Monitoring  of  Marine  Mammals  in  the  Hawaii  Range  Complex 

(HRC),  the  Gulf  of  Alaska  (GoA),  and  the  Mariana  Islands  Range  Complex  (MIRC)”  (N62470- 
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10-D-301 1  (award  to  HDR,  Inc.,  with  subcontract  to  D.  Mellinger/H.  Klinck)).  The  methods 
developed  here  are  being  applied  to  sounds  recorded  on  glider  flights  in  the  specified  areas. 

“Simple  performance-characterized  automatic  detection  of  marine  mammal  sounds”  (N39430-14-C- 
1434)  award  to  D.  Mellinger  et  al.  This  project  is  creating  a  means  to  download 
detectors/classifiers  over  the  Internet  and  use  them  in  Ishmael,  and  establishing  a  two- 
communications  channel  with  MATLAB  so  that  detection/classification  reports  made  there  can  be 
imported  back  into  Ishmael. 

“Acoustic  Metadata  Management  and  Transparent  Access  to  Networked  Oceanographic  Data  Sets” 
(NOPP  N000 14-1 1-1-0697)  award  to  PI  Marie  Roch,  Co-PI  Simone  Baumann-Pickering,  John  A. 
Hildebrand,  et  al.  A  metadata  management  system  is  being  developed,  which  allows  access  to 
locally  stored  acoustic  detections  and  metadata  and  links  in  a  standardized  way  to  external 
sources,  such  as  oceanographic  or  ephemeris  data.  We  will  design  our  DCL  plugins  to  provide 
outputs  that  can  easily  be  stored  in  the  acoustic  metadata  database. 

“PMRF  acoustic  data  collection  and  analysis”  (N000701WR4C673)  Operations  and  Maintenance 
Navy  (OMN)-funded  effort  supporting  COMPACFLT’s  Hawaii  Range  Complex  monitoring  of 
marine  mammals  during  training  activities. 

“Improving  the  Navy’s  automated  methods  for  passive  underwater  acoustic  monitoring  of  marine 

mammals”  LMR  project  PI  Tyler  Helble.  Collaboration  with  localizing  humpback  call  sequences. 

“Obtaining  cue  rate  estimates  for  some  Mysticete  species  using  existing  data”  (N0001414WX20588) 
ONR  effort  PI:  Tyler  Helble.  Collaboration  on  localization  efforts  for  cue  rate  estimation. 
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Figure  1:  Md  buzz  delected  within  the  context  of  a  foraging  dive.  Blue  dots  represent  clicks 
classified  as  Md  foraging  clicks  and  blue  x's  are  clicks  that  were  classified  as  buzz  clicks.  Green  dots 

are  clicks  that  were  (erroneously)  classified  as  Zc. 
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Figure  2.  Example  of  manually  verified  buzz  detections  from  AUTEC.  Over  a  3-day  period  in  July 
2014,  69  dives  with  1  or  more  buzzes  were  tallied,  with  an  average  of 3.696  buzzes  per  dive.  These 
results  were  particularly  encouraging  because  buzz  detections  were  received  on  hydrophone  across 
the  entire  range,  an  area  encompassing  1000+  km2.  However,  while  several  buzzes  per  dive  were 
often  detected,  these  represent  a  small  fraction  of  the  total  number  of  buzzes  produced  per  dive 

(avg.=29)  (Tyack  et  al.  2006). 
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Figure  3.  CS-SVM  output  from  7  hydrophones  for  an  exceptional  dive  encounter  involving  a  group 
of  3  vocal  Md.  Lower:  Md  foraging  click  classifications.  Magenta  squares  indicate  dive  start  and 
stop  times  automatically  determined  in  post-processing.  Upper:  Md  buzz  click  classifications  versus 
time  per  hydrophone  number.  Red  circles  indicate  buzz  start  and  stop  times  automatically 
determined  during  post-processing.  A  total  of  37  buzzes  (95%  of  the  39  manually  found)  were 
automatically  detected  during  this  complex  group  dive  event. 
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Figure  4.  Number  of  minke  whales  acoustically  localized  in  1  It  temporal  bins  for  a  study  area  of 
3, 780  km2  (PMRF)  offshore  of  Kauai,  Hawaii  ( from  Martin  et  al.  2015).  The  plots  show  the  number 
of  individual  minke  whales  (y-axis)  over  time.  “ Before ”,  “after”  and  Phase  A  and  B  refer  to  Navy 
training  exercises,  with  only  Phase  B  involving  MFA  sonar.  Blank  periods  indicate  when  recorded 
data  was  not  available.  The  reduced  numbers  in  the  phase  B  periods  are  clear. 
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